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ABSTRACT 

The influence of three sets of instructions on 
teacher/course evaluation by student raters was investigated. 
Students did not differ in their teacher/coarse evaluation ratings 
vhen the instructions specified the evaluation results vould be used: 
(1) only by the instructor; (2) by the administration; or (3) by 
students for course selection purposes. Secondarily, the effects of 
rank and class level vere also examined. It vas found that the 
evaluation of graduate courses vas significantly higher than that of 
undergraduate courses. With respect to academic rank, graduate 
teaching assistants received higher ratings than did either assistant 
or full professors. (Author/LBH) 
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The Effects of Different Instructions on Student 



Ratin/n:s of Faculty 
Venu G, Gupta, Ph.D. 
Kutztown State College 

Abstract 

The primary purpose of this study vas to investi/^ate the 
influence of three sets of instructions on teacher/coiorse evaluation 
by student raters. Students did not differ in their teacher/co\irsG 
evaluation ratings when the instructions specified the evaluation 
results would be used: (a) only by the instructor- (b) by the 
administration: or (c) by students for course selection purposes. 
Secondarily, we examined the effects of rank and class level. We 
found that the evaluation of graduate courses was siojiificantly 
hi/^er than that of underfrraduate courses. With respect to academic 
rank, rraduate teaching? assistants received higher ratinr2;s than did 
either assistant or full professors. 



The Effects of Different Instructions on Student 



Ratings of Faculty* 

Venu G. Gupta, Ph.D. 
Kutztown State College 

Statement of the problem 

It is a rare college or xiniversity that has not experimented 
with some system of student ratings of instruction. Such teacher/ 
course evaluations serve a variety of functions. Prominent among 
these functions is that of providing; administrators with comparative 
data on faculty personnel for the purposes of determining salary 
increases, promotions, and tenure. A second major function or puarpose 
of student ratings is to give teaching faculty feed-back for improving 
instruction. Still another function is to offer students information 
useful for selecting courses— if the date are published. Often, 
the same instrument serves all functions- 

Given that student ratings of instruction con serve at least 
three functions, do differences in the stated purposes for such 
evaluation influence student ratings? For example, do students rate 
professors differently if thoy ore told that the ratings will be 
used by superiors for administrative decisions than if they are 
instructed that the ratings will be used solely by the professor being 
evaluated for his own self-improvement? The primary thimst of this 
study was to compare evaluations which were obtained under three 



* This research paper was presented at the 8/.: annual convention 
of the American Psychological Association at Washington, D.C., 
September 3-7, 1976. The author gratefully acknowledges the assis- 
tance of Drs. H. Parker Blount and William M. Stallings of Georgia 
State Uni\ irsity, in the preparation of thi.s paper. 



different purpose'-instructibn conditions. 

We have been xanable to find a published study that investigated 
the effects of instructions upon student ratings. Aleamoni and 
Hexner (1973) compared the ratings of one professor xander two 
different sets of instruction. During one semester students completed 
a course /instruction questionnaire with no instructions besides those 
relating to the form. During the other semester the students vere 
told that the purpose of the instrument was both to inqprove instruction 
and to provide data which wovdd be used " for salary and pro- 
motion consideration of your instructor by his department head'^ (p. 6). 
Students who received these latter instructions rated tha professor 
significantly hif^er than those who received no instruction. An 
obvious limitation of the Aleamoni and Hexner (1973) study is that 
the improvement of instruction and administrative use purposes were 
confounded. 

In addition to the impact of instructions on student ratings, wo 
were concerned with two other variables, which might plausibly inter- 
act with ''instructions*' and which are of intrinsic interest themselves. 
Costing Greenoughj and Menges (1971) discuss several variables that 
have been sh^rwn to differentially influence evaluations by students. 
The two of particvilar interest to us were the rank of the instructor 
and the class level of the student. 

Both Downie (1952) and Gage (1961) indicated that professors 
receive higher ratings than instructors at lower ranks. Several inves- 
tigators have found that upper level students tend to assign more 
favorable ratings to their instructors than do lower level students 
(e.g., Downie, 1952 and Gage, I961). Data from the University of 



Illinois (Alearaohi:, 1972) suggest that the higher the level of the 
class, the more positive the student ratings. Some investigators, 
however, have reported that class level failed to make a difference 
in evaluations (e.g. Heilmetn & Armentrout, 1936 j Raydex, 1968). 

The purposes of this study vere to determine the influence of 
three variables (a) instruction/pxirpose, (b) academic raxik, and (c) 
class level on course /teacher evaluations. 
Method 

Subjects^. A total of kO instructors at a southern \irban xmiversity 
were evaluated by their students during a regular academic quarter. 
The proportion of instructors by academic rank and class level taught 
was: eight graduate teaching assistants teaching undergraduate classes; 
eight assistant professors teaching undergraduate classes; eight 
assistant professors teaching graduate classes: eight professors 
teaching undergraduate classes and eight professors teaching graduate 
classes. In each category but the last, the sex of instructors was 
balanced. The average number of students per. class was about 21. 

Instrument . The Illinois Course Evaluation Questionncdre 
(Aleamoni & Spencer, 1973) was the i. . 'ument used for faculty evalu- 
ation. This instrument contains 50 .1 ' rt scaled items divided into 
six subtests each of which provider^ a bUbscore. Additionally, there 
is a *'Total'' score. The ''Method of Instruction'^ subtest and the sub- 
test labeled ''Instructor" were most representative of the teaching/ 
course characteristics in which we were interested. 

We added a final item ("Person") to the instrument which read as 
follows: '^Disregarding his characteristics as a +eacher, how do you 
feel about your instructor as a person? '' The alternatives were Likert 
scaled. 
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The dependent variables vere subscores for the "Method of Instruc- 
tion"' and 'Instructor" Bubtaats, the "Total" score, and the score for 
the ''Person^' item. 

Procedure . The evaluation was carried out in class, during 
class time in the last three weeks of the quarter. The students were 
given a package which included instructions, the Illinois Course 
Evaluation Questionnaire, and a separate answer sheet. 

The experimenter J the some in each case^ passed out the packets 
to the students sequentially so that every fourth student received 
the same set of instructions, after which the experimenter gave some 
additional general instructions pertaining to the answer sheet. The 
instructions for the professor only condition were: 

The purpose of this evaluation is to provide your instructor 
with your feelings about him/her as an instructor and his/her 
course. Only the instructor will receive the restilts of this 
evaluation. The results will communicate what you perceive 
as his/her strengths and Treaknesses for the course improve- 
ment. 

The instructions for the use by the administration condition were: 
The purpose of this evaluation is to provide the University 
administration with information to make important decisions 
about each member of the faculty. The evaluation will provide 
the administration with data from which promotion^ salary- 
raises, tenure and Job retention decisions can be made. 

The instructions for the use by the students condition were: 

The purpose of this evaluation is to gather information from 
students about the courses and instructors on this coitrpus. 

er|c 



A student committee vill publish the results in a booklet con- 
taining evaluation data on all the instructors. This booklet- 
will be made available to the students in the student center 
so that they may consvtLt it in making decisions regarding 
courses and faculty selection at the time of registration. 
Results 

The unit of analysis was the "instructional set" group mean 
within each of the UO classes. That is, within each class students 
were randomly assigned to one of three sets of instructions. In 

effects we formed three groups jjer class and the mean of each group, 

I 

by class, provided the data for 'pur analysis. Since we sampled UO 
classes, we obtained a total of 120 group means. 

Correlations among the dependent variables ranged from .69 to 
.9U. This inter-dependoncy suggested that a multivariate analyses 
was appropriate. Consequently, two multivariate analysis of variance 
were calculated. 

Analysis I . The first MANcivA analysis was a three-way factorial 

I 

design with four dependent variables. For factor A (instruction 
effect) there were three levels' of instructions^ for factor B (rank 
effect) there were two levels^-professors and assistant professors; 
and for factor C (class effect) there were two levels—graduate and 
undergraduate classes. Each cell contained eight replications. 

The instruction effect and the rank effect did not reach statistical 
significance (£ ^ .05) but the class effect was found to be signifi- 
cant (2. < -05). None of the interactions was significant. 

Ob probe the significant main effect of class , univariate 3x2x2 
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analyses of variance were coiaputecL for each of the four dependent 
variables. In each analysis the F- ratios for class effect vere 
sic^ificant (to < .05) but nonsignificant for the instruction effect, 
the rank effect and the interactions (2. > •05). For each dependent 
variable, the mean for the graduate classes exceeded the nean for the 
undergraduate classes. 

Analysis 2 . The second MANOVA was a two-way factorial design 
with the same four dependent variables* This analysis compared the 
Rraducto teaching assistants, who tau^^t undergraduate classes only, 
with assistant professors and full professors, who were also teaching 
undergraduate classes. ThuSg for factor A (instruction effect) there 
were three levels of instruction. For factor B (rank effect) there 
were also three levels: professors, assistant professors, and graduate 
teaching assistants. Each cell contained eight replications* 

The instruction effect and the interaction effect were foiind to 
be statistically non-significant whereas the rank effect was found 
to be significajit at the .05 level. 

The sif^nificant rank effect suggested follow-up \miveriate 
analyses for each of the four dependent vairables. The F-ratios for 
the rank effect were significant at the .05 level, but did not reach 
significance for the instruction effect and the interaction. 

The Newman-'Keuls tests for pair-wise comparisons was cal6\ilated 
for the rank effect means. The tests indicated t>iat on the '^Total*', 
"Method of Instruction" and '^Person*^ scores the graduate teaching 
assistants received significantly higher ratings than both assistant 
professors and professors (£ < -O5). There was no significant difference 

ERIC ^ 
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between the ratings given the assistant professors and the professors. 
On the "Instructor" score ^ however, both graduate teaching assistar^ts 
and assistetnt professors were assigned significantly higher ratin/^s 
than professors (£ ^ .05); the difference in the ratings of assis- 
tant professors and the graduate teachinr; assistants was not 
statistically significant (£ > .05). 
Conclusion 

In summary, it may be stated that the intended use of the reetilts 
of the teacher/coxirse evaluation did not seem to influence the student 
evaluation. Apparently whatever set the students were using was 
sufficiently powerful to not be differentially inflvienced by the 
supposed purposes of the evaluation. The effects of instructor rank 
and class level upon student ratin^^ suggest that institutional norms 
shoTild take these two variables into account.. 
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